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Abstract 

Graduate attributes (GAs) have become a necessary framework of reference for the 21 st century competency-based 
model of higher education. However, the issue of evaluating and assessing GAs still remains unchartered territory. In 
this article, we present a criteria-based method of assessment that allows for an institution-wide comparison of the 
various acquisition levels of different GAs. In order to achieve this, we first propose an understanding of GAs as 
knowledge, skills and attitude constructs, which directly impacts the operational development of GA scales. Second, 
after briefly discussing some shortcomings in current assessment/evaluation tests for GAs, we present the many 
features of the criteria-based model for assessing GAs, such as the importance of the proper interpretation of GAs as 
can-do statements, a theory-based development of the abstract categories that make up a scale for GA assessment and 
concrete examples of GA scales based on these abstract theories. 
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1. Introduction 

Many aspects of graduate attribute (GA) assessments have been discussed in recent years, be it at the level of GA 
conceptualization, stakeholder identification and relative weighting, implementation strategies, curriculum approaches, 
staff development, quality assurance or the role of students (Hughes & Barrie, 2010). This article focuses on the 
particularly thorny issue of evaluating and assessing GAs. It presents a criteria-based method of assessment that allows 
for an institution-wide, perhaps even interinstitutional, comparison between the acquisition levels of different GAs. 

GAs are notoriously difficult to assess, and many have expressed skepticism about current attempts. In the words of 
Pitman & Broomhall (2009), 

[h]ow exactly could one measure “awareness and respect for others” - an attribute produced by as 
many universities as problem-solving skills? Can “behaving ethically” - an attribute purportedly 
inculcated in almost two-thirds of Australia’s graduates - be metricated? [...] In this formulation, 
generic skills have been transformed from a relatively universal, publicly-owned and measurable 
concept, to a personalized, unmeasurable asset’ (p. 450). 

Nonetheless, there is a general consensus that assessing GAs is essential to its successful implementation (Hughes & 
Barrie, 2010; Fraser & Thomas, 2013). This implies our taking to heart the theoretical principles of assessment. For 
instance, within a framework of assessment for learning, GAs would be best assessed in a model in which both students 
and instructors engage in self-evaluating acquisition and/or pertinence of a specific GA to the learning process where 
the criteria for each GA is detailed and explained for both stakeholders (Popham, 2007). Moreover, a proper 
assessment of GAs needs to be well anchored in the principles of assessment. In order for a systematic assessment 
process to work, it needs first to emanate from the vision and the mission that a given institution has set for itself; 
second, to be based on specific competencies that guide the learning outcomes of the institution, and; third, to involve 
strategic key stakeholders in the assessment process (Popham, 2007). These aspects will be addressed in this article. 

While the principles of assessment will be taken into consideration in this article, the primary focus will be on the 
presentation of a criterion reference model of assessment, its theoretical basis as well as its potential application for 
specific GAs. In such a model of assessment, the performance of the examinees - in the case of this article, instructor 
and students - is compared to a set of criteria defined beforehand. Linn and Gronlund (2000) define 
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Criterion-Referenced Assessment as a ‘type of assessment designed to provide a measure of performance that is 
interpretable in terms of a clearly defined and delimited domain of learning tasks’ (p. 42). Clearly and firmly defining 
the criteria that each GA measures and encompasses is at the core of the model we are proposing. 

In order to properly present the criteria-based model for assessing GAs, we will first propose an understanding of GAs 
as knowledge, skills and attitude constructs, which will have an impact on the operational development of GA scales. 
After briefly discussing some shortcomings in current assessment/evaluation tests for GAs, we will present the many 
features of the criteria-based model for assessing GAs, such as the importance of the proper interpretation of GAs as 
can-do statement, a theory-based development of abstract categories that make up a GA assessment scale, and concrete 
examples of GA scales based on these abstract theories. This, we hope, will lay a solid foundation for eventual 
implementation of GA assessment strategies. We believe it important to first clarify the conceptual prerequisites for 
assessing GAs before applying empirical methodologies (Steur, Jansen, & Hofman, 2011). 

2. Graduate Attributes 

Though there is no standard definition, GAs can be broadly defined as the qualities that assist individuals’ integration 
into both society in general and the working world after graduation (ABUS, 2004). According to the Australian 
Technology Network Report, GAs are 

the qualities, skills and understandings a university community agrees its students should develop 
during their time with the institution. These attributes include, but go beyond, the disciplinary 
expertise or technical knowledge that has traditionally formed the core of most university courses. 

They are qualities that also prepare graduates as agents of social good in an unknown future. (Boud 
& Solomon, 2006, p. 212; Bowden, Hart, King, Trigwell, & Watts, 2000) 

Some efforts have been deployed in the academic community to establish a ‘definitive’ set of GAs as a response to a 
perceived ‘inconsistency’ (Bennett, Dunne, & Carre, 1999; Sumsion & Goodfellow, 2004). This, in our minds, is 
fruitless, as GAs should be considered a response to contextual interests and pressures that, per definitionem, are 
mutable and peculiar to institutions and societies (Barrie, 2006). It makes little sense that a large institution would 
target all possible competencies at all time. First and foremost among the relevant contextual elements are the 
institution’s philosophy and values, which may differ considerably from one institution to another. In fact, cultures 
within a large institution may display significant differences: the Engineering culture and goals certainly differs from 
the Arts culture, which in turn is different from the Education culture, and so on. Faculties and departments can respond 
to different market pressures when planning the development of their students. Any GA assessment model must take 
into consideration such variances within an institution, the institution’s overarching values and the variable outcomes 
they are targeting for graduates. Hence, depending on the institution’s philosophy and values ( e.g. citizenship- or 
society-centered values vs. work- and employability-centered concerns), different sets of GAs can be espoused. For the 
purpose of this project, the GAs recently adopted by the Sub-Committee on Graduate Attributes at the University of 
Alberta will be used. Briefly, these competencies are: ethical responsibility, scholarship, critical thinking, 
communication, collaboration, creativity and confidence. 

An additional difficulty related to GA assessment has to do with the fact that they are not homogeneous. They comprise 
learning content that is referred to alternately as ‘qualities’, ‘skills’, ‘competencies’, ‘understandings’, ‘attitudes’, 
‘dispositions’, ‘values’ and so on. Indeed, GA lists adopted by institutions often include such attributes as critical 
thinking, communication skills, collaborative skills, information literacy, or even more abstract traits such as ethical 
awareness, intercultural awareness, confidence or creative thinking - concepts and mental phenomena of a different 
nature, evidently. Such diversity does not seem to lend itself to a straightforward form of assessment (Knight & Page, 
2007). However, these concepts all have the following traits in common: 

1. They can be learned or improved upon. At the very least, learnability must be postulated in the context of an 
educational institution (Kember, 2009). According to Hager (2006), some of the learning can be ‘tacit’, yet 
this should not be seen as a limitation (p. 30). Learning it is, nonetheless. 

2. They are praxis-oriented. They refer to a ‘know-how’ as opposed to a ‘know-that’ and require practice to be 
maintained (Dreyfus & Dreyfus, 1986). 

3. They are strongly contextual or situational. Indeed, GAs are qualities deemed advantageous in the 
workforce or in society, not as cultural goods that have value in themselves or for the sake of a vita 
contemplative. This is highlighted by most commentators, including Hager (2006), who believes that 
‘graduate attributes are inherently holistic and contextual in character,’ (Hager & Holland, 2006, p. 10; Hager, 
2006), and Holland (2006), for whom GAs ‘are demonstrated in situations rather than being separate entities’ 
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(p. 305). 

The first point involves the educational activity par excellence, that is, learning. Given the complexity and diversity of 
its learning contents, it is clear that GAs must encompass the whole range of learning objectives. At least since Bloom 
(1956, 1968), this range has been defined in terms of three ‘domains’ or three types of learning: knowledge (cognitive 
domain), skills (psychomotor domain) and attitudes (affective domain). Note that the ‘knowledge, skills and attitudes’ 
(KSA) triad should not be confused with the ‘knowledge, skills and abilities’ triad, often used and referred to in training 
and hiring practices, but also used in the context of GAs (e.g. Barrie, 2006, p. 217; 2007, p. 440). 

The KSA group must be adapted to our purposes if it is to serve as a framework for GAs. First, ‘knowledge’ needs to be 
conceived as praxis-oriented, in accordance with the second point of commonality. This will have consequences in the 
way the hierarchical criteria for knowledge-based GAs will be determined. Second, ‘skills’ as a psychomotor concept 
is inappropriate for GAs in the context of the university, where skills refer rather to more intellectual abilities ( i.e to 
something one can ‘do’ or accomplish with the help of one’s intellectual faculties). In this sense, written 
communication skills refer more to the intellectual ability to organize a text and convey meaning than to calligraphic 
dexterity. Third, ‘attitudes’ should be understood in its broad sense as defined by psychologists, that is, as ‘an 
evaluative judgment,’ be it feelings, a tendency, an object of memory, a categorization, etc., which ‘influence how we 
process information and how we behave’ (Maio & Haddock, 2009, p. 4). In this sense, values are also attitudes. It is 
postulated that all specific GAs are susceptible of being subsumed under a KSA category. 

3. Existing GA Tests and their Shortcomings 

The research on GAs has led to the development of evaluation models, with Australian universities no doubt leading 
the way in this respect. The University of Sydney, notably, uses the Student Course Experience Questionnaire (SCEQ) 
to collect data from students about teaching and learning goals, including generic attributes. The SCEQ comprises a 
number of items, scored on a five-point Likert scale, as well as some open-ended questions that cluster around factor 
scales, one of which is generic skills. The items relating to each scale are averaged over the respondents to produce 
scores at the faculty and university level (SECQ, 2010). 

With the signing of the Bologna Declaration in 1999, and over the course of the ensuing Bologna Process, a number of 
European countries have also pledged to implement program reforms leading to a convergence towards a European 
Higher Education Area. This will, among other things, prepare students for their future careers and for life as active 
citizens in democratic societies and support their personal development, as well as establish a norm and quality 
standards for easy movement within EU nations among the recent graduates (Bologna Process, 2010). In response to 
this process, researchers at the University of Graz developed the Graz Instrument for the Evaluation of Competencies 
(GEKo) (Dorfer & Ressler, 2009). GEKo uses the input of both students and instructors to conduct course evaluations 
that comprise different types of competencies. Through surveys and literature, the university identified five 
competence areas, each comprised of ten items. The evaluations were conducted with ordinal scales for each item. 

Other approaches have also been adopted. The Evaluation in Higher Education: Self-Assessed Competencies 
(HEsaCom) mechanism (in German: ‘Berlin Evaluation Instrument for Self-Reported Student Competencies,’ 
BEvaKomp) is based on an idea similar to GEKo (Braun & Leidner, 2009). Where traditional evaluations measure the 
satisfaction with the course or the instructor, the HEsaCom focuses on the students’ personal benefit in terms of 
competencies (Braun & Leidner, 2009). Like GEKo, this tool also breaks down the relevant competencies into 
individual items, which are then scored on ordinal scales. 

These questionnaires all have shortcomings that need to be improved upon. The SCEQ, being the oldest tool, is the 
most rudimentary. It does not indicate the level to which GAs are actually targeted in a course, as perceived by the 
instructor, leaving little way to assess to what extent course content needs to be modified from a curriculum 
management perspective. It does not give any indication of the level of acquisition of attributes, but simply ascertains a 
subjective perception of improvement. Finally, the attributes measured lack the level of complexity that should be 
expected. 

The GEKo certainly represents an improvement over the SCEQ. It is a much more complex measure of graduate 
attributes. It comprises four different questionnaires, one intended for ‘interactive’ (e.g., seminars), one for 
‘teaching-oriented’ (e.g., lecture), one for ‘practical’ (e.g., laboratory) and one for ‘e-learning’ courses. Optionally, the 
teacher can also complete the questionnaire, and in so doing establish his/her learning objectives. This methodology is 
obviously not adapted to the North American context of education, which is less rigid in its teaching formats. Moreover, 
the questions in the GEKo are often formulated in such a way that it does not relate to a student’s acquisition of 
competencies, but rather to the quality of an instructor’s teaching. 
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The HEsaCom does specifically target the acquisition of competencies but it does not provide descriptive criteria for 
the levels of acquisition, which makes it unsuitable for course comparison and curriculum management and 
development (Braun & Leidner, 2009). Our criteria-based competency model intends to improve on these existing 
tools. 

4. GA Levels of Acquisition 

The first step in establishing the model is determining the intensity levels of GA acquisition in order to set up a scale 
with intuitive labels. We have opted for a 5-point Likert scale, including a N/A entry. At level 3, mastery and/or 
acquisition of an attribute is deemed acceptable in a university setting. Levels 1-2 correspond to levels of 
pre-acquisition. Levels 4-5 designate levels of excellence that may go beyond what is expected in a university setting 
and may not be reached by all students. Given the nature and subject matter of the courses, all attributes may not be 
targeted in all courses. The N/A entry allows thus to learn more about courses and the representation of GAs in the 
courses. 

The labels given to each GA level need to be appropriately descriptive of the acquisition/mastery of each targeted GA. 
Following our understanding of GAs as KSA, this acquisition process is not limited to knowledge but also includes 
skills and attitudes. In this respect, for instance, it makes little sense to speak of ‘advanced’ levels of acquisition when 
speaking of ethical values, even though it would be appropriate with regard to acquisition of written or oral 
communication skills. The following labels were chosen because they are suited to all types of GAs while indicating a 
clear hierarchical progression: 

Table 1. GA Levels of Acquisition 


Level 

Label 

1 

Emergent 

2 

Basic 

3 

Adequate 

4 

Superior 

5 

Exceptional 


In order to measure the GAs and their indicators (identified below), we have used descriptive rating scales (Haladyna, 
1997). Known generally as the Likert rating scale, the graded-category scale allows us in this context to measure 
performance at a general yet explicit level on an interval basis. Descriptive rubrics allow us in this context to measure 
the performance on each GA, and they provide pertinent and explanatory details for each level. Because of their 
richness and ease of use, as Haladyna (1997) refers to them, the descriptive scale is the best choice for assessing 
performance: users will have a sentence-long explanation of each of the five levels within the Likert scale. 

Anticipating somewhat on a later theoretical explanation, each level describes a similar degree of learning complexity. 
At the emergent level, GA acquisition refers to the awareness of individual (or atomic) elements that are needed to 
perform a specific task, such as the cognizance of facts, ideas or rules. At the basic level, GA acquisition involves 
manipulation and combination of the basic individual elements (facts, ideas or rules) in a coherent (molecular) whole 
aimed at performing a specific task. At the adequate level, the minimum standard/norm for performing a specific task 
has been met and a GA is deemed functional in the academic context. At the superior level, a GA is acquired to the 
extent that it allows for new applications in, and generalizations to, unforeseen contexts. Finally, at the exceptional 
level, there is a consistency and spontaneity in the capacity to generalize GA application and adapt to new situations, 
including outside the academic environment. 

Each level needs to be associated with a general theoretical principle that will guide the formulation of the description 
of indicators for all GAs. This will allow for some consistency in assessing different types of GAs. We recall that, as 
learning content, GAs have been analyzed into Bloom’s three learning ‘domains’, i.e., knowledge, skills and attitudes 
(KSA). Each of these types of learning presents levels of accomplishment and complexity. Because they differ in 
nature, the description of their learning levels also has its peculiarities. Bloom (Krathwohl et ah, 1964) elaborated three 
taxonomies - cognitive (knowledge), affective (attitudes) and psychomotor (skills) - which, at first glance, seem to be 
good candidates as a basis for assessing GAs. Unfortunately, the correspondence between the types of GAs and 
Bloom’s KSA is mostly nominal. For instance, as mentioned earlier, the psychomotor domain is restricted to physical 
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skills, and cannot, as a result, apply to the intellectual skills targeted in the GAs. As for knowledge, its orientation 
toward praxis must be emphasized. For the purposes of GA assessment. Bloom’s work needs to be adapted and 
complemented. This will be the object of the subsequent sections. But before discussing this question, we need to look 
at the form taken by GA lists in institutions and how they can be operationalized to allow assessment. 

5. Interpretation of GAs 

Institutions that define graduate attributes usually set up a short list with fewer than 10 items. As a result, there is often 
the need to define a series of sub-attributes, whose number can vary from one attribute to the other. Strictly speaking, 
sub-attributes are GAs united in a subgroup defined by a common generic attribute. The following list in Table 2 below 
has been set by a student-driven initiative at the University of Alberta in 2013 (Chelen, Andrews, & Dew, 2013). The 
first column lists the generic attribute that groups the specific GAs of the second column. The third column defines a 
possible interpretation of the GAs. The formulation of a clear interpretation is crucial to establishing assessment 
criteria for the attributes and sub-attributes and usually serves as a baseline for the criterion defining the ‘adequate’ 
level of a GA acquisition. Given the many cultures within a large institution (the values and interests of the faculty of, 
say, Medicine is certainly different from those of Engineering or Arts or Education) and the different requirements of 
specific programs, there is often a need to define a specific interpretation of the sub-attributes peculiar to a faculty or a 
unit, for even though GAs may apply across the board, their interpretation may vary from program to program, from 
faculty to faculty and so forth. Whereas attributes and sub-attributes are general concepts applicable to the whole 
university, the interpretation should take on the praxis-oriented form of a can-do statement and can differ from one 
faculty to the other. 


Table 2. GA Interpretation 


Attributes 

Sub-attributes 

Interpretation 

Ethical 

responsibility 


Can adopt the perspective of moral principles rather than self-interest 


Global citizenship 

Can consider issues from a global perspective 


Community 

engagement 

Can consider issues from the perspective of their impact on the 
community 


Social and 

environmental 

awareness 

Can adopt the perspective of the public good and take into consideration 
our embeddedness within society and nature 


Professionalism 

Is willing to meet the level of expertise and deontological expectations 
of her intended profession 

Scholarship 


Can rely on a body of established knowledge to guide her action 


Knowledge breadth and 
depth 

Can make use of a broad range of knowledge while displaying mastery 
in specific areas 


Interdisciplinarity 

Can integrate into a single activity / project knowledge drawn from 
more than one academic discipline 


Life-long learning 

Is willing to engage in autonomous self-teaching in our outside the 
classroom 


Investigation 

Can effectively conduct research with the help of established methods 
and tools 

Critical thinking 


Can contextually assess given information (inch self-related) through 
reflection and debate, taking nothing for granted 


Analytic and synthetic Can gather various detailed information, organize it for specific 
reasoning purposes and assess its validity 
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Interpretive proficiency 

Can convert individual facts into meaningful information and 
knowledge 


Intellectual curiosity 

Is eager to learn beyond what is readily available (in classrooms or in 
common knowledge) 


Information literacy 

Can effectively identify and assess information within its broader 
societal contexts, inch knowledge-dependent contexts requiring 
scientific, digital or technology literacy 

Communication 


Can exchange thoughts, feelings and information effectively in various 
situations 


Writing skills 

Can write effectively in various types of writing pieces 


Oral skills 

Can speak effectively in various formal and informal settings 


Visual communication 

Can convey ideas effectively through visual aid 


Multilingualism 

Can communicate effectively in more than one language 

Collaboration 


Can complete tasks effectively by working jointly with others who share 
a common goal 


Openness to diversity 

Can engage with a diversity of people (in terms of race, religion, 
cultures, classes, sex orientation and appearance) 


Interpersonal skills 

Can demonstrate skills necessary for effective interaction and 
communication (inch empathy, active listening, respect) 


Adaptability and 

compromise 

Can change or suspend a personal belief in order to further the 
realization of a common goal or to adjust to new circumstances 


Individual contribution 

Can take an active role in collaborative work 

Creativity 


Can produce something new and valuable (inch ideas, works or 
products) 


Imagination 

Can conjure up new ideas and representations in a productive manner 


Innovation 

Can devise novel and better ways of doing things through knowledge 
(scientific, technological, methodological) 


Divergent thinking 

Can explore new avenues in a non-conformist and risk-taking fashion 


Artistic sensibility 

Can be compelled by artistic work and, ideally, partake in expressive 
artistic production 

Confidence 


Can act and think decisively 


Leadership and 

empowerment 

Can influence others into adopting an appropriate course of action 
toward a common task 


Independence 

Can work and think productively with no or little supervision 


Initiative 

Can initiate a course of action without prompting 


Resilience 

Can follow through on a course of action over time 
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Note that the GAs can be viewed from different perspectives. For instance, ‘knowledge breadth and depth’ can be 
considered an attitude, a skill or knowledge, depending on the direction one wishes to give to the understanding of this 
GA. As a result, it could mean a willingness or an eagerness to expand and deepen our knowledge based on the value 
attached to it (attitude), the capacity to expand and deepen our knowledge (skill) or the possession of broad and deep 
knowledge (knowledge). GAs can also be interpreted as both values and skills, or both skills and knowledge (Pitman & 
Broomhall, 2009). The appropriate perspective is not carved in stone, but rather depends on the intended goal, which in 
turn is conditioned by the values and goals of the institution. The appropriate understanding of the GA should be 
reflected in its interpretation (i.e. in the can-do statement). With this additional ‘hermeneutical’ layer, one overcomes 
Knight & Page’s (2007) argument that GAs may not be amenable to assessment because of their multifarious, 
underdetermined and context-dependent nature. 

Once an unequivocal interpretation of the GA has been established, a five-point Likert scale and its items can be set. 
This step of the process is contingent on the type of GA (knowledge-type, skill-type or attitude-type GA). In the 
following sections, each type of GA is associated with a theoretical framework that guides the choice indicators (the 
Likert items): Bloom’s taxonomy of the cognitive domain for knowledge-type GAs, the Dreyfus model of skill 
acquisition for skill-type GAs and Bloom’s taxonomy of the affective domain for attitude-type GAs. 

6. Knowledge-Type GAs 

When GAs correspond to knowledge, the obvious way to determine the level of acquisition is to refer to Bloom’s well 
known taxonomy of cognitive learning. In our context, the categories for assessing GAs would be set as follows: 


Table 3. Categories for Assessing Knowledge-Type GAs 


Level 

Label 

Principle 

Action 

Bloom analog 

1 

Emergent 

Knowledge of facts and ideas 

Describe, define, identify 

Knowledge 

2 

Basic 

Understanding the meaning of facts and ideas 

Explain, infer 

Comprehension 

3 

Adequate 

Using knowledge to solve problems 

Apply 

Application 

4 

Superior 

Break information into parts and patterns, and 
combining them in a different way 

Compare, distinguish, 
combine, compose 

Analysis and 

synthesis 

5 

Exceptional 

Asses and defend ideas 

Assess, evaluate, critique 

Evaluation 


For the purpose of this project, analysis and synthesis have been merged into one indicator, as they both conform to the 
general definition of the ‘superior’ level described earlier. The assessment categories (1 through 5) can serve as 
‘template’ for the actual rubrics in the criteria-based scale used for assessing specific GAs. For instance, when applied 
to the ‘knowledge breadth and depth’ sub-attribute under the ‘scholarship’ GA, we get the following criteria-based 
scale: 


Table 4. Scholarship - Knowledge Breadth and Depth Rubrics 


Can make use of a broad range of knowledge while displaying mastery in specific areas 

1 

Emergent 

Can describe a wide array of ideas and facts, some of them in great detail 

2 

Basic 

Can explain a wide array of ideas and events, some of them in great detail 

3 

Adequate 

Can use a wide array of ideas and events, some of them in great detail for specific purposes 

4 

Superior 

Can break a wide array of ideas and events into parts and patterns, some of them highly specialized, 
and combine them in a novel way 

5 

Exceptional 

Can assess the value of a wide array of ideas and events, some of them being highly specialized 


Published by Sciedu Press 


33 


ISSN 1927-6044 E-ISSN1927-6052 





www.sciedu.ca/ijhe 


International Journal of Higher Education 


Vol. 3, No. 3; 2014 


It is important to note that the wording of level 3 corresponds fairly closely to the interpretation (the can-do statement) 
of the sub-attribute ‘knowledge breadth and depth.’ All other levels (1, 2, 4 and 5) align with the categories defined in 
table 3. Note also that, in accordance with the praxis-oriented nature of GAs, emphasis has been placed on what the 
learner can ‘do’. 

7. Skill-Type GAs 

Because intellectual skills, as opposed to psycho-motor skills, are generally targeted in GAs, there is a need to look 
elsewhere than in Bloom’s taxonomy for an appropriate description of acquisition levels. Dreyfus’s model is 
particularly interesting for the context of assessing generic GAs, especially given its suitability for assessing complex 
situational skills that develop in ‘unstructured’ situations (Dreyfus & Dreyfus, 1986, p. 20). Such situations ‘contain a 
potentially unlimited number of possibly relevant facts and features, and the ways those elements interrelate and 
determine other events are unclear. Management, nursing, economic forecasting, teaching, and all social interactions 
fall into that very large class.’ Conversely, in ‘structured’ situations, the ‘goal and what information is relevant are clear, 
the effects of decisions are known, and verifiable solutions can be reasoned out’ (Dreyfus & Dreyfus, 1986, p. 20) 

As Benner (2004) puts it, the Dreyfus model ‘is situational rather than being a trait or talent model because the focus is 
on actual performance and outcomes in particular situations’ (p. 189). It follows an Instructional Design approach that 
has strong roots in the North American context. Similar elsewhere is the notion of ‘constructive alignment’ (Biggs, 
1996). The Dreyfus model assesses skills in a context of indetermination in which the required action and its context 
are not predefined: ‘As the Dreyfus model suggests, experiential learning requires the stance of an engaged learner 
rather than a stance of one expert in techne who skillfully applies well-established clear circumstances’ (Biggs, 1996, p. 
190). These traits make it a viable candidate for measuring skill-type GAs. 

Dreyfus & Dreyfus (1986) have proposed a series of five stages of skill acquisition. The first stage is the novice level. 
The novice acquires rules that s/he applies to well-defined, context-free facts, with no use of her/his judgment. Because 
the rules are not context-sensitive, there may be situations in which they will not be valid. Formal description: when 
fact/occurs, action a is required (f n —>a n ). 

The advanced beginner integrates limited situational judgment and starts to recognize patterns through similarities 
with other occurrences. However, all tasks and rules have equal weight and are not yet coordinated in a coherent pattern. 
Formal description: given situation S, fact// calls for action a h f 2 for a 2 , and so on. S(f n —>a n ). 

At the level of competence, the learner can organize situational elements (facts that make up a situation) in a 
meaningful way while assigning relative importance to them in order to reach a goal. This involves planning in a 
deliberate and reflective fashion. Formal description: given the well-defined situation S with facts /,/ and/, where/ 
is considered more or less important, action a is required in order to reach goal g. At this level, however, tasks are 
somewhat rigidly defined as routines. (S(f n )—»a n )—>g. 

At the proficient level, the learner can adapt her/his learning to new situations with the help of intuition, that is, based 
on experience and with no explicit deduction or reasoning. Patterns of tasks and rules are understood, not merely 
successions of elements, and the learner can recognize the most important and salient elements in a situation. 
Understanding becomes ‘dynamic’ as it changes according to the situation. Formal description: it ‘seems’ that situation 
S as a whole calls for actions a h a 2 and a 3 in a particular order and weighing, while the similar situation S’ calls for a 
different set of actions. (S^»ai, a 2 , a x , S’^»a 3 , a 4 , a y )—>g. 

Finally, the expert can perform all aspects of a task unconsciously, completely immersed in a situation, with no 
reference to a rule or a maxim. The learner no longer ‘applies’ rules but is rather guided by intuition. At this level, s/he 
grasps the different possible goals that can be set. Formal description: the learner ‘sees’ the salient elements of situation 
S with little or no regard to facts/,/ and/, and ‘knows’ to apply actions a 3 , a 2 and a 3 . (S^»ai, a 2 , a x ) ^»g. 

This model can serve as a basis for establishing GA assessment categories. However, there is a need for adaptation 
and/or interpretation according to the special circumstances of GAs. In particular, an assessment procedure does not 
have the luxury of asking learners for self-descriptions and draft a phenomenological account of the learner’s 
experience. Because an assessment procedure, in order to get buy-in from stakeholders, cannot be onerous, it must rely 
on observable features. Following the principles of the Dreyfus model, we propose the following categories for the 
assessment of skill-type GAs: 
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Table 5. Categories for Assessing Skill-Type GAs 

Level 

Label 

Principle 

Action 

Dreyfus 

Analog 

1 

Emergent 

Context-free knowledge of facts 
and rules 

Identify, recognize (facts and rules) 

Novice 

2 

Basic 

Contextual knowledge and 
pattern recognition 

Compare, contrast, distinguish 

Advanced 

beginner 

3 

Adequate 

Organization of contextual facts 
and rules towards a goal 

Plan, design, organize, prioritize, choose, 
infer, discuss 

Competent 

4 

Superior 

Fluid adaptation of learning 
through experience 

Anticipate, adapt, commit, engage, have 
insight 

Proficient 

5 

Exceptional 

Consistent, assured and 

spontaneous action 

Perform effortlessly, consistently and with 
assurance, critically reflects on actions 

Expert 


The regulative concept behind skill attributes is effectiveness: to possess a skill means being effective at doing 
something. However, effectiveness takes different faces depending on the task. For instance, effective writing entails 
consistent, grammatically-correct and vocabulary-rich conveyance of a message; effective critical thinking involves 
identifying and explaining problems, recognition of evidence and assumptions, implicit or explicit, evaluating and the 
assessment of implication; and so on. In the actual rubrics used for assessing a particular GA, effectiveness needs to be 
made explicit. Here is what a criteria-based scale could look like for the ‘analytic and synthetic reasoning’ sub-attribute 
under the ‘critical thinking’ GA: 

Table 6. Critical Thinking: Analytic and Synthetic Reasoning Rubrics 


Can gather various detailed information, organize it for specific purposes and assess its validity 


1 Emergent Can identify relevant information and arguments (e.g. the central problem, implicit and explicit 
assumptions) 


2 Basic Can identify alternative perspectives and justify the choice of relevant perspectives 


3 Adequate Can (re-)organize relevant information for specific purposes (e.g. reconstruct arguments) and 
evaluate assumptions (inch methodology, evidence and inference) 


4 Superior Can assess the implications and potential conclusions of an argument 


5 Exceptional Cans consistently assess the validity of information and arguments 

8. Attitude-Type GAs 

With regard to attitude-type GAs, Bloom’s taxonomy in the affective domain seems quite appropriate. Its explicit goal 
is to define hierarchical levels of thinking behaviors that relate to attitudes, feelings, values, beliefs and the like. 
According to Savickiene (2010), the most commonly referred to psychological constructs in the affective domain are 
attitudes and values. 

Bloom’s affective domain taxonomy is structured so as to describe an increasing level of internalization of intellectual 
content. The first level of affective learning, according to Krathwohl, Bloom & Masia (1964) is receiving. At this stage, 
a learner simply takes notice or is aware of an idea or phenomenon. The second level is responding, which involves 
active participation on the part of the learner in the form of reacting or responding to an idea or phenomenon. At the 
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third level, valuing, the learner assigns value or worth to an idea or phenomenon, which gives it meaning and 
importance. The fourth level is defined by organization. In this context, values are contrasted and prioritized according 
to relative importance into a coherent whole. The fifth level is called internalization or characterization. At this final 
stage, values are integrated into the learner’s character, so much so that they consistently determine behavior. 

These categories are compatible with the general GA levels of acquisition defined in table 1. ‘Receiving’ refers to the 
awareness of elemental facts and ideas, in accordance with the ‘emergent’ level of GA acquisition. ‘Responding’ 
implies the combination of an elemental fact with an appropriate action (or reaction). ‘Valuing’ is precisely, according 
to the definition of attitude given previously, what is ‘done’ when one holds an attitude and corresponds, accordingly, 
to the level of adequate acquisition of attitude-type GA. ‘Organization’ means that learners can go beyond given facts 
and ideas and generate something new - in this case, they can adopt an overview that allows for generalization. Finally, 
‘internalization’ leads to consistent and habitual behavior. 

Following Bloom’s taxonomy of the affective domain, we propose the following categories for the assessment of 
attitude-type GAs: 


Table 7. Categories for Assessing Attitude-Type GAs 


Level 

Label 

Principle 

Action 

Bloom Analog 

1 

Emergent 

Awareness 

Identify, recognize, take interest 

Receiving 

2 

Basic 

Active participation 

Discuss, explore, cooperate, volunteer 

Responding 

3 

Adequate 

Subjective involvement 

Takes a position 

Valuing 

4 

Superior 

Synthesis 

Defend, generalize, synthesize, compose 

Organization 

5 

Exceptional 

Principled, consistent behavior 

Consistently view, plan 

Internalization 


Bloom’s taxonomy of the affective domain applies quite smoothly to specific attitude-type GAs. Flere is an example of 
a criteria-based scale for the attitude-type sub-attribute of ‘social and environmental awareness’ under the ‘ethical 
responsibility’ GA: 

Table 8. Ethical Responsibility: Social and Environmental Awareness Rubrics 


Can adopt the perspective of the public good and take into consideration our embeddedness within 
society and nature 

1 

Emergent 

Displays and awareness of social and environmental issues 

2 

Basic 

Shows a willingness to participate in discussions on environmental issues 

3 

Adequate 

Adopts behaviour that demonstrates strong beliefs in and sensitivity toward environmental issues 

4 

Superior 

Can explain and/or defend one’s own beliefs and involvement in environmental issues 

5 

Exceptional 

Displays an ingrained commitment for environmental issues 


Following the interpretation (can-do statements) given for each GA and categories for assessing knowledge-, skills- 
and attitude-type GAs (tables 3, 5 and 7), criteria-based scales can be created for all GAs. 


9. Conclusion 

The focus of this article was primarily on presenting a criteria-based model for assessing GAs, which still needs to be 
tested for validity and reliability in an empirical setting. This model is founded on the understanding of GAs as KSA 
(knowledge, skills and attitudes), a tripartite categorization that encompasses the many forms taken by GAs. It also 
relies on the idea that GAs need to be ‘interpreted’ as praxis-oriented can-do statements. These two measures allow for 
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subsequent operationalization, a crucial step prior to assessment. Emphasis was placed on the theoretical basis that 
supports the development of assessment scales, thus allowing for a flexible framework for creating new scales for any 
additional GA. Each KSA type rests on a particular theoretical frame (Bloom-cognitive, Dreyfus model. 
Bloom-affective) while displaying a common core. 

In itself, this is but a first step in the task of assessing GAs. What is still required is an implementation model that 
establishes a proper assessment procedure that takes into account the needs, interest and concerns of all stakeholders 
and fulfills the pragmatic condition of successful application. These steps should provide the proper groundwork for 
the subsequent empirical study of GAs. 
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